A New Method for Variable Elimination in Systems 

of Inequations 

Farhad Shirani Chaharsooghi, Mohammad Javad Emadi, Mahdi Zamanighomi and Mohammad Reza Aref 

Information Systems and Security Lab (ISSL) 
Electrical Engineering Department, Sharif University of Technology, Tehran, Iran. 
Email: fshirani@ee.sharif.edu, emadi@ee.sharif.edu, m_zamani@ee.sharif.edu, aref@sharif.edu 



Abstract — In this paper, we present a new method for variable 
elimination in systems of inequations which is much faster than 
the Fourier-Motzkin Elimination (FME) method. In our method, 
a linear Diophantine problem is introduced which is dual to our 
original problem. The new Diophantine system is then solved, 
and the final result is calculated by finding the dual inequations 
system. Our new method uses the algorithm Normaliz to find 
the Hilbert basis of the solution space of the given Diophantine 
problem. We introduce a problem in the interference channel 
with multiple nodes and solve it with our new method. Next, 
we generalize our method to all problems involving FME and in 
the end we compare our method with the previous method. We 
show that our method has many advantages in comparison to 
the previous method. It does not produce many of the redundant 
answers of the FME method. It also solves the whole problem 
in one step whereas the previous method uses a step by step 
approach in eliminating each auxiliary variable. 

I. Introduction 

The FME Method was first introduced by Fourier [_] in 
1827 and was rediscovered in Motzkin's thesis in 1936 _3. 
Considering the set of inequations Ax < b,A G R m,n , b G R n 
the method provides a scheme for determining the existence 
of a real or integer solution to the given problem. The method 
uses a step by step algorithm, in which each step reduces the 
dimension of the answer space by projecting the answer space 
on a hyperplane. The procedure continues until the dimension 
is reduced to one; if the one dimensional answer is feasible, 
then the solution exists. A dual mode for FME was introduced 
in 1972 by Dantzik __. 

The FME method is widely used in solving rate region 
problems since there is often the need to eliminate the auxil- 
iary variables introduced into the problem from the resulting 
inequations and hence reduce the dimension of the solution 
space by finding its projection on a given plane. The com- 
plexity of calculation in each step is of the order of 
with k being the number of constraints in the system of 
inequalities. This complexity makes the method unfeasible for 
high dimensional problems. There have been many attempts to 
decrease the calculation time of the method. In 1996, Kebler 
[4 1 introduced a parallel computation method which was more 
efficient when solving the elimination problem on a computer. 

In this paper, we present a new method for eliminating 
auxiliary variables in inequations which is much faster than 
the FME method. In our method, we first introduce a linear 
Diophantine problem which is dual to the original problem. 
We then proceed by solving this new Diophantine system, next 



we calculate the final result by finding the dual inequations 
system. Our new method uses the algorithm Normaliz to find 
the Hilbert basis of the solution space of the given Diophantine 
problem. 

Normaliz was first introduced as a program in 1997 by 
R. Koch. In 2003, the algorithm was presented in Koch's 
thesis __. Normaliz was originally introduced to compute the 
lattice points in a lattice polytope and the integral closure of a 
monomial ideal. Koch mentions in his thesis that the algorithm 
may be extended to find the Hilbert basis of the solution space 
of a system of linear Diophantine equations. However, apply- 
ing the algorithm introduced in __ to information theoritic 
problems is still time-consuming. In 2010, Bruns and Ichim [6] 
introduced a dual algorithm which is much faster when applied 
to problems with a greater number of constraints than auxiliary 
variables. Since this is the case in almost all information theory 
problems, we use this method to find the desired Hilbert Basis 
solution. 

One of the most important fundamental channels in com- 
munication systems is the Interference Channel (IC). The IC 
was initiated by Shannon [7]. Simple outer bounds for the IC 
were established by Ahlswede __. Later, Carlieal [9], using 
superposition encoding and sequential decoding, characterized 
an achievable rate region for general discrete memoryless IC. 
In the IC each receiver is only interested in the message from 
its corresponding sender. However, it might decode messages 
from other senders to reduce the interference effect. Han 
and Kobayashi (HK) utilized rate splitting techniques in 1 10], 
and applied simultaneous superposition coding techniques to 
establish the largest inner bound introduced for the general IC 
to date. Computing the capacity region of general IC is still an 
open problem, but in special cases such as strong interference 
regime the capacity region has been established, and the region 
was the same as the HK rate region ifTTI . In the method used 
to achieve the HK rate region, the messages are split into 
two parts, the private and common parts; private messages 
are decoded by the corresponding receiver, while the common 
messages are decoded by all receivers. 

Rate splitting has become a prominent tool for solving rate 
region problems. However, there is a drawback to the method 
in that solving the channel with the new split messages yields 
a system of inequations based on the split-rates. Since we are 
interested in each sender's total rates, the auxiliary variables 
must be eliminated. The FME method is needed to eliminate 



these auxiliary rates. 

The rest of the paper is organized as follows: In Section 
II, we first introduce our model of an I transmitter-receiver 
interference channel. In Section III, we introduce a problem in 
the IC. In this section, we use some simplifying assumptions, 
since the goal of the section is to explain the new elimination 
method. In Section IV, we solve the problem with our new 
method. In Section V, we present a generalized algorithm 
for our new method which could be applied to any problem 
involving FME. In Section VI, we present results of applying 
our new method to several problems which were solved 
previously using the FME method. Section VII concludes the 
paper. 

II. Channel Moddel 

In this section, we introduce our model of IC. We denote 
random variables by upper case letters e.g. X, their realizations 
are presented by lower case letters e.g. x in finite sets denoted 
by X. We use x\ to denote the vector X4, X4+1, £fc. We 
denote x k if i=l, while for i=l and k=/, we show the vector 
by x. Also, if i=l and k=2l, we show the vector by X- 
Furthermore xf is the compact form of {x i ,x i+1 , ...,x n ) 
where x { = (xa, x a ). 

A discrete Interference Channel, with I senders and I 
receivers is the 21+1 tuple (X,p(y\x), f ) where Xi are the 
input alphabets, % are the output alphabets for i=l,2,...J 
and p(y\x) is the conditional probability of the channel. 
Each decoder is only interested in decoding its corresponding 
sender's message; however, in order to reduce the interference 
effect, receivers might decode the message from other senders 
as well. The channel is memoryless and time invariant in the 
sense that 

p(y n k n >y n ^)^PY\x(y n k n ). 

A ((2 nRl ,2 nIi2 ,...,2 nR '),n) code consists of (i) I message 
sets Wi = 1, 2, 2 nRi , i G [1 : I], (ii) I encoding functions 
/„ = W u — > X™,u G [1 : I], (iii)/ decoding functions <?„(.) 
where W u = g u (Y™) : u G [1 : /]. We define the average 
probability of error for this code as P e (n) = Pr{{j l i=1 {W t ^ 
Wi)}. A nonnegative rate Z-tuple (Ri, R 2 , Ri) is said to 
be achievable if there exists a code ((2 nRl , 2 nR * , 2 nRl ), n) 
for which P e (n) -> as n tends to infinity. 

III. Problem Statement 

In this section, we present our coding strategy which is 
a generalization of the method used in [10|. We describe 
the problem in solving the resulting inequations with the 
conventional FME method. After stating the problem, we 
introduce our method and compare the results with the former 
procedure. 

Since our goal in this paper is to introduce an alternative 
elimination method for the FME method, we use some sim- 
plifying assumptions in the channel. We consider the channel 
to be symmetric in the sense that the zth sender sends the 
same amount of information to all receivers other than its 
own corresponding receiver. This allows each sender to send 



the same common message to all receivers. We split each 
message Wi into common and private parts (Wj c , Wi P ). The 
common part of the message is decoded by all receivers while 
the private part is decoded only by the ith receiver. Another 
suitable assumption would have been to assume a sense of 
degradedness between the receivers in the following way: 
assume that receiver i\ receives all the common massages sent 
by sender i; receiver i 2 receives all the common messages 
received by the receivers other than i\, and so on. In this case 
we had to split the message in each sender to I common parts 
and one private part in the following way: split the message 
Wi into (Wi(i c ), Wi( 2c ), -, Wi(i c ), W ip ). Wi(i c ) is decoded by 
the receiver with the least information about the massage from 
sender i; the reciever with the second least information about 
this message decodes Wia c -\, Wj(a_ n c ), and so on. The rest 
of the solution would stay the same. Clearly, the symmetric 
case mentioned above is a special case of this latter case. 
Since the two cases have the same solution, and for the sake 
of simplicity, we solve the channel for the symmetric case in 
this paper. 

Codebook Generation: 

Let Z 1 = (U 1 ,U 2 ,...,U l ,X 1 ,X 2: ...,X l ,Y 1 ,Y 2 ,...,Y l ,Q), 
where Q is the time-sharing variable. Let 2>i denote the set of 
all jont p.m.fs pi on Z\, which can be written in the form 

l 

p( z i) =p(q) Y\[p{ui\q)p{xi\ui,q))p{yi,y 2 , ...,yi\x 1 ,x 2 , ...,x t ) 

For any p\ defined above, generate 2 nRic ,i G [1 : I], i.i.d 
u ic( w ic) where w ic G [1 : 2 nRic ]. For each uf c (wi c ), generate 
2 nRi p, i.i.d x2(wip,Wi C ). The time-sharing variable has no 
impact on the rest of the proof and for the sake of brevity, 
we ignore it until the end of the proof where we bound its 
cardinality. 
Encoding: 

At the beginning of each block, the ith sender picks related 
{uf c ,xf) according to Y[ n j=l p{u ic ^)p{x ltj \u iCtj ) and sends it. 
Decoding: 

At the end of each block, the ith receiver finds 

(xf, u" c , u^ c , w" c ) such that: 

(xf(wi p ,w ic ),u^ c (wi c ),u 2c (w 2c ),...,uf c (wi c )) G A e 

Without loss of generality we consider the case where all 
senders have sent index 1 as their messages. We investigate 
all possible scenarios in the first decoder. The decoder must 
decode messages (wip,wi c ,w 2c , Wi c ), the decoding is er- 
rorless if ii — (wi p , wic) = (1,1)- Using the same method as 
in lfT2l and by the AEP rule, it is clear that the probability 
of error approaches zero as n — >• 00 (e.g. rate /-tuple is 
achievable) iff the rates satisfy the following set of equations: 

Rip + a-iRic + a 2 R 2c + ... + aiR\ c < h,ai,a 2 ,— ,a t (1) 
where 

a 4 G {0,1} 



and Ui G Us if oii — 1; also, for a, = 0, we have [/j G L/gc. 
We note that R\ — R\ c — R\ v , substituting into ([T]) we have 



R\ -aiRi c + a 2 R2 



-aiRi c < Ii >ai , 



,(* G{0,1} 



These are the inequations for decoder 1. We rewrite the 
inequations for all decoders in matrix form: 
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If we denote A as the matrix whose rows are all possible binary 
/-tuples starting from (0, 0, 0) in an increasing order, then 
Ai is the matrix formed by multiplying the ith column in A 
by -1. We show the rate coefficients matrix in (f2) with C;. 
We show the rate vector with R , and the mutual information 
vector with so we may write (f2]i in compact form as Qi? < 
Ii. To clarify the method of forming the above mentioned 
inequations, we give the following example, which illustrates 
the inequations for a two sender-receiver interference channel: 
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Note that these are the same inequations given in lfl2l . To 
solve this system with the conventional FME method, one must 
eliminate all of the variables Ri C , i G [1 : I]; this process takes 
I rounds of applying FME to the matrix. According to \12\, 

.2 

the size of the matrix grows at a worst case rate of 4- each 
round. This complexity renders the aforementioned method 
impractical for I larger than 2, since, after applying FME, 
one must eliminate the redundant inequations in the resulting 
matrix using an exhaustive search. Finding the redundant in- 
equalities often needs more time than eliminating the auxiliary 
variables, which itself is a time-consuming process. 



IV. Our New Method 

Here, we present a new method for solving (f2) which not 
only requires much less calculation in comparison with the 
FME but also produces no redundant inequations assumning 
the mutual-informations on the right side of the inequations are 
independant. Before proceeding with the solution, we present 
some definitions. We show the matrixes resulting from I rounds 
of FME by C* and /,*, so the resulting inequations can be 
written in the form CfR_ < If . Note that the second / columns 
of C ; * are zero. It is also clear that if we show the rows of C; 
by C;(z ai a2 m \, then all rows of C* and /* can be shown 

by 
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l.Ql ,a2,...,CK; 
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a, G {0, £ [1 : l],a k{i 
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Let Gc t be the solution space of the inequations in (fj) for 
rates i? 2 , Ri, and let Gq* be the solution space of 
C*R < I*. Furthermore, we define the matrix D as the matrix 
whose rows are formed by all of the linear combinations of 
Cj(j >ai iQ2 a A with positive coefficients and with the property 
that the second I columns of D are zero. In other words, D. is 
a row of D iff 

12. ^ . fl i,ai,a2,...,QiC!(i,a 1 .Q2,.,.,ai) (3) 

with the following conditions: 

^fc(i,Ql ,Q2,...,Q() — 



DfU = 0; 



(4) 



We define Gd, as the solution space of the following inequa- 
tions: 

Dig < I Dl (5) 



where 



2,Ctl ,CK2,..-,CK; 



Theorem 1: Gc* is equal to Gjj r 

Proof: The proof is straightforward. Since all rows of 
Di are linear combinations of rows of C\ with positive 
coefficients, all inequations in (O result from inequations in 
d2}- Hence, we have Gc t Q Gdi> an( J by the FME, we 
know that Gc, is equal to Gc* for {R±, R 2 , Ri), we have 
Gc* Q Gd 1 ■ On the other hand, since all the rows of C* are 
linear combinations of rows of C\ with the property that for 
each row G*, C*f +1 = 0, any row C_* of C* is a row of Di, 
and so Gd { Q Gc* and the proof is complete. ■ 



Now, we propose that instead of finding Gc* , we try to find 
Gdi ■ From |@), we know that for every D in (0) we have 
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— aj,ai : a 2: ... : a J _i,l,Q i+ i,...,Qi (6) 

ai,£«2i— ,ay_i,ay+i,...,ai 

Now we need to choose inequations in © whose linear 
combinations with positive coefficients produce all inequations 

in ©. Without loss of generality, we consider aj j0ll ,a 2 a t S 

Z + . We give the following definition as in lfl3ll : 
Definition: We call a set of vectors H, generator vectors of S, 
if every vector in S can be expressed as a linear combination 
of vectors in H with positive coefficients and no vector in H 
may be expressed in such a way by other vectors in H. Such 
a set of vectors is called the Hilbert Basis of the vector space 
S. 

The problem of finding a Hilbert Basis for the solutions 
of a Diophantine system of equations such as (|6]l has many 
solutions. One solution is to use an algorithm called Normaliz 
introduced by Koch in f5). We use this method to solve ((6]). If 
the set of Oj ) a 1 ,a 2 ,...,aj> j = lj —,1,0^ € {0, 1} is an answer 
for ©, then we have the following inequation for R\ , . . . , Ri : 

i 



3=1 



,3 



,ai ± J.a 1 ,a 2 ,...,ai 



Since Normaliz provides a Hilbert Basis solution for the 
equations in ([6j, and since each solution for (|6]l represents a 
unique equation in ((5]) and adding two answers from ((6) with 
positive coefficients is equivalent to adding two inequations in 
(O with positive coefficients, the resulting inequations contain 
all the non-redundant inequations in Q. Hence, by applying 
Normaliz, we can calculate the solution to our original prob- 
lem. Since the dimension of the Diophantine space in (O is 
much greater than the number of constraints we have, it is 
better to use Normaliz Dual as suggested in J6). Our results 
for the two sender-receiver IC are the same as HK rate region 
in HI. 

To this point, we have found the rate region by eliminating 
the auxiliary variables. Since, in the case of the above channel, 
all the mutual-informations Iiu a i,a^,...,ai) m ' e independent of 
each other (i.e. they don't have a clear relation with each 
other), there are no redundant inequations in our final system 
of inequations and so by the Caratheodory theorem [14] Q is 
bounded by the number of resulting inequations after applying 
Normaliz and finding the dual system. 

V. Generalization to Other Problems 

The method proposed above may be used in any other 
problem which could be solved using the FME. The following 
algorithm is used: 

1- Define the encoding and decoding schemes. 

2- Find the inequations matrix corresponding to the defined 
schemes. 



3- Write the Diophantine equations of the matrix. 

4- Use Normaliz to solve the Diophantine problem. 

5- For each answer vector for the Diophantine problems, write 
the corresponding inequation. 

VI. Simulation and Results 

Here, we use our method to solve some problems which 
have been solved using the FME method previously. The 
simulation results are illustrated in (|l): 

TABLE I 

Results of Applying Our New Method on Some Examples 
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The results are obtained from running Normaliz 2.2 on 
a 2.4 i5 CPU. The 4th column indicates the number of 
inequations resulting from applying our method while the 5th 
column indicates the number of non-redundant constraints. 
Note that in cases such as [ 15 1 and 1 16] where there are certain 
relations between mutual-informations on the right side of the 
inequations, there may be redundant answers in our method. 
However all these redundant answers will be present in the 
FME case as well. Table (U) shows the efficiency of our method 
in comparison to the FME method. There are several features 
of our method that make it more efficient in comparison 
with the former method. One feature is that the FME method 
eliminates one variable at a time, while our method eliminates 
all variables in one step. Another feature is that our method 
is much faster compared to the previous method. Another 
important advantage of the new method is that it doesn't 
introduce many of the redundant inequation present in the 
FME solution. Note that one of the most time-consuming 
parts of the FME method is the removal of inequations which 
are linear combinations of previous inequations with positive 
coefficients. This process uses an exhaustive search and is very 
inefficient. Our method doesn't introduce any such redundant 
answers. 

VII. Conclusion 

In this paper, we introduced a new method for performing 
variable elimination in systems of inequations. We illustrated 
the implementation of our new method on a special case of IC. 
A generalization of the method was presented and in the end 
some simulation results were compared with the actual results 
after applying FME and removing redundant answers. It was 
shown that our new method is much faster and more efficient 
than the FME method. Our method eliminates all variables in 
one step while the FME method uses a step by step approach. 



Also, our method produces fewer redundant inequations in 
comparison to the FME method. 
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